Method and apparatus for compressing coding unit in high efficiency video coding

ABSTRACT

A method for decoding of a video bitstream including receiving coded data for a 2N×2N coding unit (CU) from the video bitstream, selecting one or more first codewords according to whether asymmetric motion partition is disabled or enabled, selecting one or more second codewords when a size of the 2N×2N CU is equal to a smallest CU size, wherein none of the second codewords corresponds to INTER N×N partition when N is 4, determining a CU structure for the 2N×2N CU from the video bitstream using the first codewords or the second codewords, and decoding the video bitstream using the CU structure. A corresponding method for encoding a 2N×2N coding unit of video data is also disclosed.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 14/527,769, filed on Oct. 30, 2014, which is adivisional application of U.S. patent application Ser. No. 13/272,221,filed on Oct. 13, 2011, entitled “Method and Apparatus for CompressingCoding Unit in High Efficiency Video Coding” (now. U.S. Pat. No.9,049,452, issued on Jun. 2, 2015), which claims priority to U.S.Provisional Patent Application Ser. No. 61/508,825, filed on Jul. 18,2011, entitled “Method and syntax for compressing coding units in HEVC”.The present invention is also related to U.S. Non-Provisional patentapplication Ser. No. 13/012,811, filed Jan. 25, 2011, entitled“Apparatus and Method of Constrained Partition Size for High EfficiencyVideo Coding”. The U.S. Provisional Patent Application and U.S.Non-Provisional Patent Applications are hereby incorporated by referencein their entireties.

FIELD OF THE INVENTION

The present invention relates to video processing. In particular, thepresent invention relates to method and apparatus for compressing codingunits in High Efficiency Video Coding (HEVC).

BACKGROUND AND RELATED ART

HEVC (High Efficiency Video Coding) is an advanced video coding systembeing developed under the Joint Collaborative Team on Video Coding(JCT-VC) group of video coding experts from ITU-T Study Group. In HEVC,a 2N×2N coding unit can be hierarchically partitioned into a partitiontype selected from 2N×2N, 2N×N, N×2N and N×N. The coding system uses acriterion to determine the best partition, where RD-rate is often usedas the criterion. The N×N partition at level k is evaluated and the samepartition, i.e., 2N×2N partition is also evaluated at level k+1.Therefore, N×N partition at level k becomes redundant if 2N×2N partitionat level k+1 will be evaluated. In order to eliminate the aboveredundancy, the allowable partition sizes are constrained according tothe method disclosed in U.S. Non-Provisional patent application Ser. No.13/012,811, filed Jan. 25, 2011, entitled “Apparatus and Method ofConstrained Partition Size for High Efficiency Video Coding”. In U.S.Non-Provisional patent application Ser. No. 13/012,811, for each leaf CUlarger than the SCU (smallest CU), the partition sizes allowed are2N×2N, 2N×N and N×2N. In other words, the N×N partition is not allowedfor INTER mode if the leaf CU is larger than the SCU. If the leaf CUsize is the same as SCU size, all partition sizes, 2N×2N, 2N×N, N×2N,and N×N, are allowed. While the method disclosed in U.S. Non-Provisionalpatent application Ser. No. 13/012,811 reduces computational complexityat the expense of modest performance loss, it is desirable to develop amethod and apparatus that can further reduce the computationalcomplexity with about the same performance. Furthermore, it is desirableto provide flexibility so that either the method and apparatus withfurther complexity reduction can be selected or an alternative methodand apparatus can be selected.

BRIEF SUMMARY OF THE INVENTION

A method and apparatus for decoding of a video bitstream are disclosed.In one embodiment, a method for decoding of a video bitstream includesreceiving coded data for a 2N×2N coding unit (CU) from the videobitstream, selecting one or more first codewords according to whetherasymmetric motion partition is disabled or enabled, selecting one ormore second codewords when a size of said 2N×2N CU is equal to asmallest CU size, wherein none of said one or more second codewordscorresponds to INTER N×N partition when N is 4, determining a CUstructure for said 2N×2N CU from the video bitstream using said one ormore first codewords or said one or more second codewords, and decodingthe video bitstream using the CU structure.

In another embodiment, a method for processing coding units of videodata comprises receiving input data associated with a 2N×2N coding unit(CU), selecting one or more first codewords according to whetherasymmetric motion partition is disabled or enabled, selecting one ormore second codewords when a size of said 2N×2N CU is equal to asmallest CU size, wherein none of said one or more second codewordscorresponds to INTER N×N partition when N is 4, determining a CUstructure for said 2N×2N CU, and encoding the CU structure using saidone or more first codewords or said one or more second codewords.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary coding unit partition based on thequadtree.

FIG. 2 illustrates allowed partition sizes of prediction unit for a2N×2N leaf coding unit.

FIG. 3 illustrates an example of redundancy problem for prediction unitat depths k and k+1.

FIG. 4 illustrates an example of constrained partition set for a 2N×2Nleaf coding unit to avoid redundancy for INTER prediction.

FIG. 5A illustrates an example of coding unit partition at variousdepths according to an embodiment of the present invention, where INTERN×N is not allowed for depth=3.

FIG. 5B illustrates an example of coding unit partition at variousdepths according to an embodiment of the present invention, where INTERN×N is allowed for depth=3.

FIG. 6 illustrates an example of sequence level syntax to supportselection of coding unit structure and associated processing.

FIG. 7 illustrates an example of picture level syntax to supportselection of coding unit structure and associated processing.

FIG. 8 illustrates an example of coding unit prediction mode andpartition mode specification for coding unit size larger than thesmallest coding unit size.

FIG. 9 illustrates an example of coding unit prediction mode andpartition mode specification for coding unit size equal to the smallestcoding unit size and the N×N partition is allowed for INTER mode.

FIG. 10 illustrates an example of coding unit prediction mode andpartition mode specification for coding unit size equal to the smallestcoding unit size and the N×N partition is not allowed for INTER mode.

DETAILED DESCRIPTION OF THE INVENTION

During the encoding process, in order to achieve the best possibleperformance, the rate-distortion function or other performance criterionusually is evaluated for various coding unit (CU) partitions andprediction unit (PU) partitions. The PU design in the current HEVCdevelopment results in some redundancy to cause rate-distortion functionor other performance criterion repeatedly evaluated for some PUconfiguration. For example, redundancy may exist between theconfiguration of the INTER N×N CU at depth=k and the configuration ofthe INTER 2N×2N CU at depth=k+1. The redundancy will cause unnecessaryprocessing and waste valuable system resources. A method to alleviatethe redundancy is disclosed in U.S. Non-Provisional patent applicationSer. No. 13/012,811, filed Jan. 25, 2011, entitled “Apparatus and Methodof Constrained Partition Size for High Efficiency Video Coding”, where aconstrained CU partition has been developed to eliminate or reduce theredundancy in processing. Nevertheless, it is desired to develop codingunit compression method to further reduce the computational complexity.Also it is desirable to provide flexibility so that either the methodand apparatus with further complexity reduction can be selected or analternative method and apparatus can be selected. Furthermore, it isdesired to design necessary syntax to convey the information related tothe efficient and flexible partition between an encoder and a decoder.

In the high efficiency video coding (HEVC) system under development, thefixed-size macroblock of H.264/AVC is replaced by a flexible block,named coding unit (CU). FIG. 1 illustrates an exemplary coding unitpartition based on a quadtree. At depth 0, the initial coding unit CU0,112 consisting of 64×64 pixel, is the largest CU (LCU). The initialcoding unit CU0, 112 is subject to quadtree split as shown in block 110.A split flag 0 indicates that the underlying CU is not split and, on theother hand a split flag 1 indicates the underlying CU is split into foursmaller coding units CU1, 122 by the quadtree. The resulting four codingunits are labeled as 0, 1, 2 and 3 and each resulting coding unitbecomes a coding unit for further split in the next depth. The codingunits resulted from coding unit CU0, 112 are referred to as CU1, 122.After a coding unit is split by the quadtree, the resulting coding unitsare subject to further quadtree split unless the coding unit reaches apre-specified smallest CU (SCU) size. Consequently, at depth 1, thecoding unit CU1, 122 is subject to quadtree split as shown in block 120.Again, a split flag 0 indicates the underlying CU is not split and, onthe other hand a split flag 1 indicates the underlying CU is split intofour smaller coding units CU2, 132 by the quadtree. The coding unit CU2,132, has a size of 16×16 and the process of the quadtree splitting asshown in block 130 can continue until a pre-specified smallest codingunit is reached. For example, if the smallest coding unit is chosen tobe 8×8, the coding unit CU3, 142 at depth 3 will not be subject tofurther split as shown in block 140. The collection of quadtreepartitions of a picture to form variable-size coding units constitutes apartition map for the encoder to process the input image areaaccordingly. The partition map has to be conveyed to the decoder so thatthe decoding process can be performed accordingly.

Besides the concept of coding unit, the concept of prediction unit (PU)is also introduced in HEVC. Once the splitting of CU hierarchical treeis done, each leaf CU is subject to further split into prediction units(PUs) according to prediction type and PU partition. For temporalprediction, the PU types consist of SKIP, MERGE and INTER modes. Forspatial prediction modes, the PU type consists of INTRA mode. For each2N×2N leaf CU, one partition size is selected. When the PredMode(Prediction Mode) is SKIP or MERGE, the only allowed PartSize (PartitionSize) is {2N×2N}. When the PredMode is INTER, the allowed PartSize isselected from the set {2N×2N, 2N×N, N×2N, N×N} as shown in FIG. 2. Whenthe PredMode is INTRA, the allowed PartSize is selected from the set{2N×2N, N×N}. The PU design in the current HEVC development results insome redundancy. For example, redundancy may exist between theconfiguration of “the PU of the CU with depth=k, Mode=INTER,PartSize=N×N” and the configuration of “the PU of the CU with depth=k+1,Mode=INTER, PartSize=2N×2N” as shown in FIG. 3. The PU 310 at depth kwill be processed again at depth (k+1) as the PU 320. The PU 310 isselected under the INTER mode with partition size N×N. On the otherhand, the PU 320 is selected at the INTER mode with partition size2N′×2N′, where 2N′=N. Consequently, the same block will be processedtwice at depths k and depth (k+1). The redundancy will cause unnecessaryprocessing and waste valuable system resources.

In order to eliminate the above redundancy, the allowable partitionsizes are constrained according to U.S. Non-Provisional patentapplication Ser. No. 13/012,811, as shown in FIG. 4. Consequently, foreach leaf CU larger than SCU (smallest CU), the partition sizes allowedare 2N×2N, 2N×N and N×2N. In other words, the N×N partition is notallowed for INTER mode if the leaf CU is larger than SCU. If the leaf CUsize is the same as SCU size, all partition sizes, 2N×2N, 2N×N, N×2N,and N×N, are allowed. When a CU size is the same as SCU size, the CU isnot subject to further split and the inclusion of N×N partition sizewill not cause redundancy. The partition types according to current HEVCHM3.0 (HEVC Test Model version 3.0) described above are summarized inTable 1. The codeword table associated with various partition types forHEVC HM3.0 is shown in Table 2.

TABLE 1 INTER INTER INTRA INTRA Partition Types CU > SCU CU = SCU CU >SCU CU = SCU 2Nx2N Yes Yes Yes Yes Nx2N Yes Yes No No 2NxN Yes Yes No NoNxN No Yes No Yes

TABLE 2 Partition type CU > SCU CU == SCU INTER 2Nx2N 1 1 INTER Nx2N 0101 INTER 2NxN 001 001 INTER NxN 0001 INTRA 2Nx2N 000 00001 INTRA NxN00000

While the method disclosed in U.S. Non-Provisional patent applicationSer. No. 13/012,811, uses constrained PU partition to reduce the codingredundancy, the process can be further improved. According to oneembodiment of the present invention, the N×N coding mode is removed forINTER coding at all depths. FIG. 5A illustrates allowed INTER and INTRApartitions in various depths according to an embodiment of the presentinvention. The example shown in FIG. 5A still allows INTRA N×N partitionwhen the CU size equals to the smallest size. Since the codeword tabledoes not need to accommodate an entry for INTER N×N regardless whetherCU is larger than SCU or CU has the same size as SCU, the codeword tablecan be simplified. An exemplary codeword table incorporating anembodiment according to the present invention is shown in Table 3. Thecodewords for INTRA 2N×2N and INTRA N×N in Table 3 are shorter than therespective codewords in Table 2.

TABLE 3 Partition type CU > SCU CU == SCU INTER 2Nx2N 1 1 INTER Nx2N 0101 INTER 2NxN 001 001 INTRA 2Nx2N 000 0001 INTRA NxN 0000

In another embodiment according to the present invention, the system canadaptively eliminate the INTER N×N partition and the selection can beindicated by syntax. For example, the sequence parameter set (SPS) andpicture parameter set (PPS) syntax can be modified to allow more codingflexibility. FIG. 5B illustrates allowed INTER and INTRA partitions invarious depths where INTER N×N partition is allowed when the CU sizeequals to the smallest size. Exemplary SPS and PPS syntaxesincorporating an embodiment according to the present invention are shownin FIG. 6 and FIG. 7 respectively. In order to provide more codingflexibility, a flag “disable_inter_4×4_pu_flag” is added in SPS ashighlighted in FIG. 6. In addition, a flag “disable_inter_4×4_pu_pic”may be added in PPS as highlighted in FIG. 7 to allow the encoder toselectively enable the INTER N×N when INTER N×N is allowed as indicatedby “disable_inter_4×4_pu_flag” in SPS. If “disable_inter_4×4_pu_flag” is1 in SPS, the INTER N×N (N=4) is disabled for the whole sequence. The“disable_inter_4×4_pu_pic” in PPS will not be sent in this case.Otherwise, the “disable_inter_4×4_pu_pic” in PPS will be sent todetermine whether to allow INTER N×N for CU=SCU is disabled for eachpicture. Therefore, if “disable_inter_4×4_pu_flag” is true, then Table 3will be used for all Inter frames in the sequence; otherwise, if“disable_inter_4×4_pu_pic” is true, then Table 3 will be used for thecurrent Inter frame, if “disable_inter_4×4_pu_pic” is false, Table 2will be used. The exemplary syntax design in FIG. 6 and FIG. 7 are forthe purpose to illustrate one means to practice the present invention. Askilled person in the field may use other syntax design to practice thepresent invention without departing from the spirit of the presentinvention. For example, instead of “disable_inter_4×4_pu_flag”, a flag“enable_inter_4×4_pu_flag”, “inter_4×4_enabled_flag” or any otherequivalence in SPS may also be used. Similarly, instead of“disable_inter_4×4_pu_pic”, a flag “enable_inter_4×4_pu_pic”,“inter_4×4_enable_pic”, or any equivalence in PPS may also be used.

The coding tree semantics associated with the syntax described above areillustrated in FIG. 8 through FIG. 10. FIG. 8 illustrates specificationof cu_split_pred_part_mode when CU is greater than SCU, wherecu_split_pred_part_mode specifies split_coding_unit_flag and, when thecoding unit is not split, the skip_flag, the merge_flag, PredMode andPartMode of a coding unit. FIG. 9 illustrates specification ofcu_split_pred_part_mode when CU is equal to SCU. In FIG. 9, INTER N×N isallowed. FIG. 10 illustrates specification of cu_split_pred_part_modewhen CU is equal to SCU and INTER N×N is not allowed, i.e.,disable_inter_4×4_pu_flag=1 or disable_inter_4×4_pu_pic=1 according toexemplary syntax disclosed above.

When Asymmetric Motion Partitioning (AMP) is enabled, additionalpartitions including INTER 2N×nU, INTER 2N×nD, INTER nL×2N and INTERnR×2N, will be used. The codeword tables in Table 2 and Table 3 can bemodified to accommodate the additional partitions as shown in Table 4,where the differences from Table 2 and Table 3 are shown in Italic.

TABLE 4 CU == SCU CU == SCU Partition type CU > SCU inter_4x4 disabledinter_4x4 enabled INTER 2Nx2N 1 1 1 INTER 2NxN 011 011 011 INTER 2NxnU0101 0101 0101 INTER 2NxnD 0100 0100 0100 INTER Nx2N 0011 0011 0011INTER nLx2N 00101 00101 00101 INTER nRx2N 00100 00100 00100 INTER NxN0001 INTRA 2Nx2N 000 0001 00001 INTRA NxN 0000 00000

In U.S. Non-Provisional patent application Ser. No. 13/012,811, filedJan. 25, 2011, entitled “Apparatus and Method of Constrained PartitionSize for High Efficiency Video Coding”, it has been demonstrated thatthe method based on constrained partition size can noticeably reduce therequired computations at the expense of very modest increase inRD-rates. The method incorporating an embodiment according to thepresent invention further selectively removes INTER N×N partition forall CU sizes to reduce computational complexity. Again, the increase inRD-rates is very modest. In another embodiment according to the presentinvention, a flag in SPS and/or PPS is used to select whether INTER 4×4is allowed. If INTER 4×4 is allowed, the coding method for CU/PUpartition similar to that of U.S. Non-Provisional patent applicationSer. No. 13/012,811 is selected. If INTER 4×4 is not allowed, the methodwith further reduced computational complexity as disclose herein isused.

Embodiment of compressing CU partition with INTER 4×4 removed accordingto the present invention as described above may be implemented invarious hardware, software codes, or a combination of both. For example,an embodiment of the present invention can be a circuit integrated intoa video compression chip or program codes integrated into videocompression software to perform the processing described herein. Anembodiment of the present invention may also be program codes to beexecuted on a Digital Signal Processor (DSP) to perform the processingdescribed herein. The invention may also involve a number of functionsto be performed by a computer processor, a digital signal processor, amicroprocessor, or field programmable gate array (FPGA). Theseprocessors can be configured to perform particular tasks according tothe invention, by executing machine-readable software code or firmwarecode that defines the particular methods embodied by the invention. Thesoftware code or firmware codes may be developed in differentprogramming languages and different format or style. The software codemay also be compiled for different target platform. However, differentcode formats, styles and languages of software codes and other means ofconfiguring code to perform the tasks in accordance with the inventionwill not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departingfrom its spirit or essential characteristics. The described examples areto be considered in all respects only as illustrative and notrestrictive. The scope of the invention is, therefore, indicated by theappended claims rather than by the foregoing description. All changeswhich come within the meaning and range of equivalency of the claims areto be embraced within their scope

The invention claimed is:
 1. A method for decoding of a video bitstream by a video decoding circuit, the method comprising: receiving coded data for a 2N×2N coding unit (CU) from the video bitstream; selecting one or more first codewords according to whether asymmetric motion partition is disabled or enabled, wherein none of said one or more first codewords corresponds to INTER N×N partition; selecting one or more second codewords when a size of said 2N×2N CU is equal to a smallest CU size, wherein none of said one or more second codewords corresponds to the INTER N×N partition when N is 4; determining a CU structure for said 2N×2N CU from the video bitstream using said one or more first codewords or said one or more second codewords; and decoding the video bitstream using the CU structure.
 2. The method of claim 1, further comprises: receiving an indication from the video bitstream regarding whether the asymmetric motion partition is disabled or enabled.
 3. The method of claim 1, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, and INTER 2N×N partition when the asymmetric motion partition is disabled, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled.
 4. The method of claim 1, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, INTER 2N×N partition when the asymmetric motion partition is disabled and the size of said 2N×2N CU is larger than the smallest CU size, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled and the size of said 2N×2N CU is larger than the smallest CU size.
 5. The method of claim 1, wherein said one or more first codewords and said one or more second codewords are used for entropy decoding based on CABAC (Context-Adaptive Binary Arithmetic Coding) or CAVLC (Context-Adaptive Variable-Length Coding).
 6. A method of processing coding units of video data by a video encoding circuit, the method comprising: receiving input data associated with a 2N×2N coding unit (CU); selecting one or more first codewords according to whether asymmetric motion partition is disabled or enabled, wherein none of said one or more first codewords corresponds to INTER N×N partition; selecting one or more second codewords when a size of said 2N×2N CU is equal to a smallest CU size, wherein none of said one or more second codewords corresponds to the INTER NxN partition when N is 4; determining a CU structure for said 2N×2N CU; and encoding the CU structure using said one or more first codewords or said one or more second codewords.
 7. The method of claim 6, further comprises: determining whether the asymmetric motion partition is disabled or enabled; and transmitting an indication in a video bitstream regarding whether the asymmetric motion partition is disabled or enabled.
 8. The method of claim 6, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, and INTER 2N×N partition when the asymmetric motion partition is disabled, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled.
 9. The method of claim 6, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, INTER 2N×N partition when the asymmetric motion partition is disabled and the size of said 2N×2N CU is larger than the smallest CU size, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled and the size of said 2N×2N CU is larger than the smallest CU size.
 10. The method of claim 6, wherein said one or more first codewords and said one or more second codewords are used for entropy encoding based on CABAC (Context-Adaptive Binary Arithmetic Coding) or CAVLC (Context-Adaptive Variable-Length Coding).
 11. A non-transitory computer readable medium storing a computer-executable program, the computer-executable program, when executed, causing a decoder to perform the following steps: receiving coded data for a 2N×2N coding unit (CU) from the video bitstream; selecting one or more first codewords according to whether asymmetric motion partition is disabled or enabled, wherein none of said one or more first codewords corresponds to INTER N×N partition; selecting one or more second codewords when a size of said 2N×2N CU is equal to a smallest CU size, wherein none of said one or more second codewords corresponds to the INTER N×N partition when N is 4; determining a CU structure for said 2N×2N CU from the video bitstream using said one or more first codewords or said one or more second codewords; and decoding the video bitstream using the CU structure.
 12. The non-transitory computer-readable medium of claim 11, wherein the decoder is further configured by the computer-executable program to receive an indication from the video bitstream regarding whether the asymmetric motion partition is disabled or enabled.
 13. The non-transitory computer-readable storage medium of claim 11, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, and INTER 2N×N partition when the asymmetric motion partition is disabled, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled.
 14. The non-transitory computer-readable medium of claim 11, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER N×2N partition, INTER 2N×N partition when the asymmetric motion partition is disabled and the size of said 2N×2N CU is larger than the smallest CU size, wherein said one or more first codewords correspond to INTER 2N×2N partition, INTER 2N×N partition, INTER 2N×nU partition, INTER 2N×nD partition, INTER N×2N partition, INTER nL×2N partition, and INTER nR×2N partition when the asymmetric motion partition is enabled and the size of said 2N×2N CU is larger than the smallest CU size.
 15. The non-transitory computer-readable medium of claim 11, wherein said one or more first codewords and said one or more second codewords are used for entropy decoding based on CABAC (Context-Adaptive Binary Arithmetic Coding) or CAVLC (Context-Adaptive Variable-Length Coding). 